In a previous work we developed a function capable of generating a M-by-N matrix of Zsurf values that, seen as a surface, contain certain patterns or shapes whose size (width and height) vary according to a normal distribution and whose distance between center-to-center (pitch) along of the X and Y axes also follows a normal distribution. Now, similar surfaces are generated from certain sources and stored in particular files with an .sdf extension, which continue a certain header configuration and in the writing of the z values. Then, this document explains how these surfaces can be replicated starting from an .sdf file and using the previously developed function. We present four new functions that we have developed, with which we managed to read those files and compute the random distributions associated with the patterns to finally use the previous function and thus replicate the surface under study.

Python Tool Developed

We have developed the code for this project in both Python and MATLAB. In this report we will see the mode of use and the functionality of the Python version. First, we will start by importing the tool which we have named sd_tool_3d, making mention of 3D Shape Distribution Tool.

In this tool, you will find all the functions available to make the project work. Once we do the import, we continue with the exiclation of the kernel functions.

In [1]:
import sd_tool_3d as imgp
import numpy as np

Now we present the four functions that we have developed to extract the statistical distributions associated with the patterns/shapes distributed on the surface. We will be presenting in a systematic order according to the procedure used to achieve our goal. We then will start with the function that performs the reading of the .sdf files until we reach the main function that returns the parameters of the desired distributions.

read_sdf function

We were looking for some function already developed that was able to read this type of files but we did not find it. Then we decided to develop it. We will let the function explain itself with its header comment.

In [2]:
#Determines the size of each of the shapes (widths and heights).
#
#Description:
#The function internally runs an iterative clustering algorithm to extract all 
#the points of each sahpe starting from the centroid. The algorithm iteratively
#selects point rings from the centroid, computes the average value (corresponding 
#to the average height of the ring) and then computes the distance of the current 
#ring from the previous one. The reference or stop criterion is the distance of
#the first ring with respect to the centroid. If the distance between the current 
#ring and the previous one is less than that reference, then the grouping ends. 
#Then the last ring found corresponds to the width of the current pattern under 
#study. Finally, the width of the shapes is the diameter of that last ring
#and the heights are simply the evaluation of Zsurf in the centroid. 
#
#Inputs:
#
#-Zsurf: M-by-N matrix of Zsurf values that, seen as a surface, contain     
#       certain patterns or shapes whose size (width and height) vary ac-  
#       cording to a normal distribution and whose distance between center 
#       to center (pitch) along of the X and Y axes also follows a normal  
#       distribution    
#       
#- pts: centroid points of each of the patterns in Zsurf.
#
#Outputs:
#
#-wc: Width of each of the shapes                                           
#-hc: Height of each of the shapes                                          
#-pc
#

Next we show that the function works well. For this (and henceforth) we will use the files called test_JP2.sdf

In [3]:
# Path to test file
filepath = "../inputs/test_JP1.sdf"

# Reading data set
_,_,Zsurf = imgp.read_sdf(filepath) 

# Domain
no, nx, ny = 450, 650, 650
Zsurf = Zsurf.astype(np.float32)
Zsurf = Zsurf[no:nx,no:ny] #define a work domain

# get the meshgrid
Y,X = np.meshgrid(range(0,ny-no+1),range(0,nx-no+1)) 

#show the original 3D surf
imgp.graph_3d(X,Y,Zsurf)

NOTE: We have selected a reduced work domain of 200-by-200 to be able to visualize the surfaces well. Then we select the domain within the total surface to avoid the edges

Fast peak function

This function is used to find the centroids of the patterns/shapes on the surface. Initially we thought about thresholding Zsurf and using the MATLAB regiongrous funtion. However, by testing we saw that regiongrous had a low success rate, leaving many centroids undetected. Then we decided to look for a more robust way and we got this function. We will let the function explain itself with its header comment.

In [4]:
#inputs:
#
#- d: The 2D data raw image.
#- thresh: A number between 0 and max(raw_image) to remove background
#- filt: A filter matrix used to smooth the image. The filter size should correspond the charactistic
#size of the peaks.
#-edg: A number > 1 for skipping the first few and the last few 'edge' pixels
#-res: A handle that switches between two peak finding methods:
#    1 the local maxima method (default)
#    2 the weighted centroid sub-pixel resolution method.
#
#-fid: (path.mat) In case the user would like to save the peak positions to a file
#
#Outputs:
#    detected_peaks: logical matrix with dimension equal to the image, in the position where 
#    there is a peak the value will be True, otherwise it will be False.
In [5]:
# Execute the function fast_peak_find
detected_peaks = imgp.fast_peak_find(Zsurf, np.median(Zsurf), 1.5)

# The centroids are
x, y = np.nonzero(detected_peaks)
centers = np.array([[x_,y_] for x_,y_ in zip(x,y) ])

# height of each center
hcenters = np.zeros(len(x))
count=0
for x_,y_ in zip(x,y):
    hcenters[count] = Zsurf[x_,y_]
    count +=1
    
print(f"Num Peaks: {count}")

#graph
imgp.graph_3d_peaks(X,Y,Zsurf,x,y,hcenters)

Num Peaks: 205

With simple visual inspection, we can see that a high percentage of real centroids were identified (almost all centroids. 205 are detected, to be exact).

get_features Function

his function stems from the search for a clustering algorithm to extract patterns from previously found centroids. But it was difficult to find an already developed function that did it, because they all use a stop criterion such as the number of samples in the groups or the maximum distance between the points. That does not help us because precisely the patterns have variable size. Then we decided to develop our own clustering algorithm based on the height of the shapes (we only work for now with top hemispheres). In this way, the function takes advantage of the sweep for each centroid and extracts the features of the forms such as: width and height. It also returns coordinate matrices that will later help us find the distribution parameters for center-to-center distances between patterns. We will let the function explain itself with its header comment.

In [6]:
#Determines the size of each of the shapes (widths and heights).
#
#Description:
#The function internally runs an iterative clustering algorithm to extract all 
#the points of each sahpe starting from the centroid. The algorithm iteratively
#selects point rings from the centroid, computes the average value (corresponding 
#to the average height of the ring) and then computes the distance of the current 
#ring from the previous one. The reference or stop criterion is the distance of
#the first ring with respect to the centroid. If the distance between the current 
#ring and the previous one is less than that reference, then the grouping ends. 
#Then the last ring found corresponds to the width of the current pattern under 
#study. Finally, the width of the shapes is the diameter of that last ring
#and the heights are simply the evaluation of Zsurf in the centroid. 
#
#Inputs:
#
#-Zsurf: M-by-N matrix of Zsurf values that, seen as a surface, contain     
#       certain patterns or shapes whose size (width and height) vary ac-  
#       cording to a normal distribution and whose distance between center 
#       to center (pitch) along of the X and Y axes also follows a normal  
#       distribution    
#       
#- pts: centroid points of each of the patterns in Zsurf.
#
#Outputs:
#
#-wc: Width of each of the shapes                                           
#-hc: Height of each of the shapes                                          
#-pc

Next we show that the function works well.

In [7]:
# Execute get_features function
wc,hc,pc = imgp.get_features(Zsurf,centers)

# Show only the patterns extrated
Zs = np.full_like(Zsurf,np.nan)
ind = np.zeros(np.shape(Zsurf));
Ys,Xs = np.meshgrid(range(0,200), range(0,200));

for i in range(0,np.shape(wc)[0]):
    xo = round(centers[i,0]);
    yo = round(centers[i,1]);
    r = wc[i] / 2;
    ind += (Xs-xo)*(Xs-xo) + (Ys-yo)*(Ys-yo) <= r**2

Zs[ind.astype(dtype=bool)] = Zsurf[ind.astype(dtype=bool)]

#graph
imgp.graph_3d_peaks(Xs,Ys,Zs,x,y,hcenters)

get 3D pattern statistics function

We have developed this function to provide a main function that encapsulates the two previous functions and returns the parameters of the random distributions associated with the patterns/shapes. Specifically, this function locates the centroids of the shapes by executing the FastPeakFind function and then runs the get_features function to obtain the values of widths, heights and coordenate matrices. Finally, on the last values returned, calculate the distribution parameters. We will let the function explain itself with its header comment.

In [8]:
    #Description.
    #
    #     receives an M-by-N matrix of Zsurf values that, seen as a surface, contain   
    #     certain patterns or shapes whose size (width and height) vary according to   
    #     a normal distribution and whose distance between center to center (pitch)    
    #     along of the X and Y axes also follows a normal distribution. Then the func- 
    #     tion, also knowing as input the type of patterns (pattype, top hemispheres   
    #     for example) contained in Zsurf, locate these patterns and extract their     
    #     main statistics. That is, it determines the normal distributions associated  
    #     with its size and pitch.                                                     
    #
    #     Inputs:   
    #
    #       -Zsurf: M-by-N matrix Z values read from an .sdf file with the special     
    #               function read_sdf().                                               
    #
    #      -pattype: string indicating the type of patterns contained in Zsurf (See    
    #                table below)                                                      
    #
    #     Outputs:   
    #
    #      -polyshape:    Dictionary with::      
    #                     - First key is a string specifying the type of 3D shape de- 
    #                       sired (see the table below to know the types of shapes     
    #                       available).                                                
    #                     - Second key is a dictionary with W_statistics                         
    #                     - Third key is similar to the previous, but with H_statistics                                   
    #
    #
    #      -distparams:   dictionary of four elements                                   
    #      -imsize:       Dictionary of two elements specifying the size of the 3D out-    
    #                     put surface, 'img'.                                          
    #
    #      -features:    dictionary of four keys.                 
    #
    #      Table: pattype options                                                      
    #      +------------+-------------+--------------+---------+-----------------+     
    #      | shape      |  long form  |  short form  |  size   |  description    |     
    #      +------------+-------------+--------------+---------+-----------------+     
    #      +------------+-------------+--------------+---------+-----------------+     
    #      | hemisphere | 'htop'      |  'ht'        | [w, h]  | w: width        |     
    #      | top        |             |              |         | h: height       |     
    #      +------------+-------------+--------------+---------+-----------------+     
    #      | hemisphere | 'hbottom'   |  'hb'        | [w, h]  | w: width        |     
    #      | bottom     |             |              |         | h: height       |     
    #      +------------+-------------+--------------+---------+-----------------+

In the next section, we will test this function by trying to replicate the original 3D surface.

In [12]:
# We knows that the pattern type is hemisphere top
pattype = 'htop'

# Execute the main function
polyshape, distparams, imsize, features = imgp.get_3d_pattern_statistics(Zsurf,pattype)

# Execute the function of previus work
img, pitchp, wsp, hsp =  imgp.shape_placement_3d_v1(imsize,polyshape,distparams)

#Get only the Z values
Zp = img[2,:,:]

# Graph Original surf
imgp.graph_3d(X,Y,Zsurf,title='Original surf')

# Graph Replicate surf
imgp.graph_3d(X,Y,Zp+np.min(Zsurf),title='Replicate surf V1')